This was a group project for university where we were tasked with finding a novel method for optimising the weights and biases of a neural network that did not rely solely on convnetional gradient descent methods and also used population-based algorithms.

My group and I came up with a hybrid algorithm which utilised Grey Wolf Optimiser hybridised with Genetic Algorithm during the mutation phase of the lifecycle to explore the decision space. We then combined this explorational approach with teh exploitative approach of Stochastic Gradient Descent to converge on any optima found in the decision space. After a set amount of epochs during the training process, we would iteratively switch in and out the two algorithms to exploit, then explore until it would converge on an optima.

Our novel approach to this task helped us achieve the highest grade in the cohort and incentivised us to publish a paper based on this research with the approval of our lecturers.

The "No Free Lunch" theorem should always be taken into acount when applying optimisation algorithms to different fitness functions. It means that no individual algorithm can be seen as being objectively better than any other for every optimisation task.